Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[rubrik] Enable TSDB #12917

Merged
merged 14 commits into from
Mar 4, 2025
Merged

[rubrik] Enable TSDB #12917

merged 14 commits into from
Mar 4, 2025
+42 −5

Conversation

gpop63
Copy link
Contributor

@gpop63 gpop63 commented Feb 27, 2025 •
edited
Loading

Overview

This PR enables:

  • TSDB for current Rubrik data streams and
  • removes the set for @timestamp in the node_statistics data stream because the time can fall outside the range of the indices resulting in documents being dropped.
  • @tommyers-elastic suggested a better approach than just removing the set for @timestamp. Instead, we will send a time range interval to the GraphQL query, which will get the data for the previous hour every hour. This ensures we are always staying in the indices' start and end times and avoiding the issue with timestamp out of range.
TSDB test 

You're testing with version 8.17.2.

Testing data stream metrics-rubrik.node_statistics-default.
Index being used for the documents is .ds-metrics-rubrik.node_statistics-default-2025.02.27-000001.
Index being used for the settings and mappings is .ds-metrics-rubrik.node_statistics-default-2025.02.27-000001.

The time series fields for the TSDB index are: 
        - dimension (4 fields):
                - rubrik.cluster.id
                - rubrik.cluster.name
                - rubrik.cluster.type
                - rubrik.node_statistics.node_id
        - gauge (8 fields):
                - rubrik.node_statistics.cpu_stat.pct
                - rubrik.node_statistics.iops.reads
                - rubrik.node_statistics.iops.writes
                - rubrik.node_statistics.network.received.bytes
                - rubrik.node_statistics.network.transmitted.bytes
                - rubrik.node_statistics.throughput.read.bytes
                - rubrik.node_statistics.throughput.write.bytes
                - rubrik.node_statistics.used_memory.pct
        - routing_path (4 fields):
                - rubrik.cluster.id
                - rubrik.cluster.name
                - rubrik.cluster.type
                - rubrik.node_statistics.node_id

Index tsdb-index-enabled successfully created.

Copying documents from .ds-metrics-rubrik.node_statistics-default-2025.02.27-000001 to tsdb-index-enabled...
All 528 documents taken from index .ds-metrics-rubrik.node_statistics-default-2025.02.27-000001 were successfully placed to index tsdb-index-enabled.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

  • [ ]

How to test this PR locally

Related issues

Screenshots

@gpop63 gpop63 self-assigned this Feb 27, 2025
@gpop63 gpop63 added the Team:Obs-InfraObs Label for the Observability Infrastructure Monitoring team [elastic/obs-infraobs-integrations] label Feb 27, 2025
@gpop63 gpop63 marked this pull request as ready for review February 27, 2025 09:50
@gpop63 gpop63 requested a review from a team as a code owner February 27, 2025 09:50
ritalwar
ritalwar reviewed Feb 27, 2025
View reviewed changes
Copy link
Contributor

@ritalwar ritalwar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe that since the dimensions were set in previous PRs, thorough testing has already been conducted to ensure no document drop, and this PR is simply enabling time series.

ritalwar
ritalwar approved these changes Feb 27, 2025
View reviewed changes
Copy link
Contributor

@ritalwar ritalwar left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please review the pipeline failures; everything else LGTM!

@elastic-vault-github-plugin-prod
Copy link

elastic-vault-github-plugin-prod bot commented Feb 27, 2025 •
edited
Loading

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

@elasticmachine
Copy link

💚 Build Succeeded

History

cc @gpop63

@elastic-sonarqube Elastic SonarQube
Copy link

Quality Gate failed Quality Gate failed

Failed conditions
0.0% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube

@gpop63 gpop63 merged commit 236d1d1 into elastic:main Mar 4, 2025
6 of 7 checks passed
@elastic-vault-github-plugin-prod
Copy link

Package rubrik - 0.8.0 containing this change is available at https://epr.elastic.co/package/rubrik/0.8.0/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@devamanv devamanv devamanv approved these changes

@ritalwar ritalwar ritalwar approved these changes

@tommyers-elastic tommyers-elastic tommyers-elastic approved these changes

Assignees

@gpop63 gpop63

Labels
Team:Obs-InfraObs Label for the Observability Infrastructure Monitoring team [elastic/obs-infraobs-integrations]
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@gpop63 @elasticmachine @devamanv @ritalwar @tommyers-elastic